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Abstract 

Introduction: Hyperspectral imaging lias been used in dernnatology for nnany years. 
The enrichment of hyperspectral imaging with image analysis broadens considerably 
the possibility of reproducible, quantitative evaluation of, for example, melanin and 
haemoglobin at any location in the patient's skin. The dedicated image analysis 
method proposed by the authors enables to automatically perform this type of 
measurement. 

Material and method: As part of the study, an algorithm for the analysis of 
hyperspectral images of healthy human skin acquired with the use of the Specim 
camera was proposed. Images were collected from the dorsal side of the hand. The 
frequency X of the data obtained ranged from 397 to 1030 nm. A total of 4'000 2D 
images were obtained for 5 hyperspectral images. The method proposed in the 
paper uses dedicated image analysis based on human anthropometric data, 
mathematical morphology, median filtration, normalization and others. The algorithm 
was implemented in Matlab and C programs and is used in practice. 

Results: The algorithm of image analysis and processing proposed by the authors 
enables segmentation of any region of the hand (fingers, wrist) in a reproducible 
manner. In addition, the method allows to quantify the frequency content in 
different regions of interest which are determined automatically. Owing to this, it is 
possible to perform analyses for melanin in the frequency range A^g(450,600) nm 
and for haemoglobin in the range ^^^(397,500) nm extending into the ultraviolet for 
the type of camera used. In these ranges, there are 189 images for melanin and 126 
images for haemoglobin. For six areas of the left and right sides of the little finger 
(digitus minimus manus), the mean values of melanin and haemoglobin content 
were 17% and 15% respectively compared to the pattern. 

Conclusions: The obtained results confirmed the usefulness of the proposed new 
method of image analysis and processing in dermatology of the hand as it enables 
reproducible, quantitative assessment of any fragment of this body part. Each image 
in a sequence was analysed in this way in no more than 100 ms using Intel Core i5 
CPU meO (5)2.5 GHz 4 GB RAM. 

Keywords: Hyperspectral imaging. Image processing. Measurement automation. 
Segmentation 
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Introduction 

The use of imaging in dermatological diagnosis is currently a very rapidly growing 
branch of medicine and computer science. Computer-assisted medical diagnosis gives 
much wider possibilities than the methods of traditional evaluation performed by a 
medical diagnostician. The results obtained from computer analysis are automatic, re- 
producible, calibrated and independent of human factors related to both the patient 
and the medical diagnostician who performs the examination. The most common 
methods of imaging are infrared and visible light. In this respect, the methods of assess- 
ment of various types of dermatological conditions based on simple methods of image 
analysis and processing are extremely popular. In thermal imaging, temperature 
changes resulting from the imaged diseases such as melanoma as well as thermal 
effects of the performed dermatological treatments are analysed automatically. As 
for imaging in visible light, a group of methods which enable morphometric mea- 
surements or profiled methods of image analysis and processing are used, for ex- 
ample, to determine the brightness of the RGB components in a segmented skin 
area, Hyperspectral imaging, which is also used in dermatology, offers much wider 
capabilities, Multispectral images are acquired using profiled multispectral cameras 
working in different spectra ranges. Additionally, depending on the frequency and 
spectrum range, various types of illuminators are used. Matching these two ele- 
ments (camera and illuminator) is extremely important because of the need to ob- 
tain a flat spectrum of the illuminator (lamp) in the range covered by the camera. 
In this range of light, for example, the visible one, it is possible to observe melanin 
or haemoglobin at a desired location on the skin. This observation is associated 
with image analysis enabling automatic and reproducible measurements independ- 
ent of interindividual variability. 

The methods of image analysis and processing used in hyperspectral imaging of 
any object (not necessarily a medical one) can be divided into several groups, 
namely morphological methods [1-6], statistical methods [7-9] and profiled algo- 
rithms for selected applications. Among the morphological methods, there are clas- 
sical approaches [1,2,4] and those profiled to the analysis of image sequences [5,6], 
The statistical methods are dominated by texture analysis [7-9] which is used as a 
set of features for classification and recognition. So far, profiled algorithms have 
been applied to face recognition [10], analysis of skin areas [11], and others [12]. 
These methods mainly dominate in the segmentation of specific objects [13]. On 
the basis of segmented objects, their morphometric measurements or analysis of 
their texture are performed [9]. These are, for example, methods which enable to 
assess the difference in absorption of radiation by skin chromophores such as hae- 
moglobin or melanin. 

The amount and distribution of melanin is in fact an important factor determining, 
inter alia, the efficacy and safety of treatments in aesthetic medicine with the use of la- 
sers and ILP. In hyperspectral imaging of melanin and haemoglobin, only qualitative 
analyses of the results obtained are known [14-16]. Therefore, there is a need to object- 
ify the results obtained and to propose fully- automatic measurements of progression 
and changes of melanin or haemoglobin in the skin of the hand. 

The aim of the analysis is to determine melanin and haemoglobin quantity in selected 
areas of the right hand by using hyperspectral imaging. 
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Material 

As part of the study, an algorithm for the analysis of hyperspectral images of human, 
healthy skin acquired with the use of the Specim PFD-VlOE camera was proposed. Im- 
ages were derived from the human hand having 2 Fitzpatrick skin phototypes. Individ- 
ual hands were illuminated with a typical lamp with flat spectral characteristics in the 
required range (based on HgAr emission for the VNIR spectral range). The images 
were obtained retrospectively during routine medical examinations carried out in ac- 
cordance with the Declaration of Helsinki. As for the described algorithm, no studies 
and experiments were carried out on humans. The resulting data were anonymized and 
stored in the output format, source 'dat' (EN VI File). The frequency A of the data ob- 
tained ranged from 397 to 1030 nm. Each image was recorded every 0.79 nm, which in 
total gave 800 2D images for each patient. The resolution MxN (number of rows and 
columns) of each image for the selected frequency was 899 x 1312 pixels. For the fixed 
distance of the object from the camera and the set focusing parameters, there was a 
square area covering the range 130 x 130 \im per each pixel. A total of 4'000 2D images 
were obtained for 5 hyperspectral images. These images are subject to further analysis. 



Method 

Preprocessing 

Input files with the extension *.dat containing a sequence of images are reorganized in 
the preliminary stage. This process (Figure 1) concerns the change in the organization 
of individual rows and columns obtained for different wavelengths to the sequence of 
images Lg]^aAj^>J^>^)> where m and n are row column coordinates and the wavelength A 
respectively, for which the image was acquired. The number of rows and columns and 
the wavelength for each image are automatically read from the file *.hdr. This file also 
contains information about the data format (e.g. data type 4 - float 32 bits) and other 
elements that need to be taken into account when reading such data (wavelength units. 




m=1 



m=2l 



times. 




*,datfile 



one line 

Figure 1 Diagram of the acquisition and organization of data ^dat. The various stages of data analysis 
and reorganization are performed automatically. A hyperspectral camera saves each row sequentially for 
individual wavelengths X. In subsequent stages of processing, they are converted to the image L^f^Mi^P,^)- 
The resulting image is further subjected to further processing steps. 
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default bands, sensor type, interleave). For the analysed data, the image resolution MxN 
was 899 X 1312 pixels. 

Each of the images Lg]^ay{j^>J^M is calibrated using the reference rows (used for the 
calibration for a given range of the spectrum A) [17-19], and then subjected to filtration 
thus forming a new sequence of images L^(m,f2,A). This is median filtration with a 
mask Hm sized 3x3 pixels. The mask size was chosen arbitrarily taking into account 
the possible contamination of the optical path as well as the image resolution and the 
size of the object [20-22]. A larger size of the mask Hm caused the removal of not only 
noise but also small elements in the image which are substantively important. Conse- 
quently, the size of 3 x 3 pixels remained. The final step of image pre-processing in- 
volves normalization and removal of uneven illumination. In the case of normalization 
of brightness levels, the range of minimum and maximum values in the image L^(m,f2,A) 
is extended to the full range of brightness from 0 to 1, that is, the image after 
normalization Lp{m,n,X) is equal to: 

LM{m,n,X)- mmyyi^n{LM{m,n,X)) 

Lp{m,n,X)= T— : -— -— (1) 

m2iXm,ri[LM{m,n,X) - mm^,^(lM(^, ^, A))) 

It should be noted that this normalization also covers the area to be calibrated. For 
this reason, the maximum brightness value equal to "1" after normalization corre- 
sponds to 100% emission for the wavelength A. Removal of uneven illumination is re- 
lated to the subtraction from the image Lp{m,n,X) the image resulting from its 
filtration, i.e.: 

Lc{m, n,X) = Lp{m, n,X) 

(2) 

for m^{MH2l% Mc'Mh2l2) and ne{Nh2l2, Nc'Nh2/ 2), where: 
Mh2 X ^h2 resolution of the mask h2, 

Lc{yn,n,X) - output image after the removal of uneven illumination. 

To acquire the output image Lc{m,n^), its convolution with the mask /z2 sized x ^h2 
equal to 30 pixels was used. The mask size was selected on the basis of twice the maximum 
size of the elements, objects visible in the image Lc{m,n,X), in this case it is 15 x 15, 
i.e. Mpi2 X ^h2 = 30 X 30 pixels. Next, based on the images Lc{m,n,X) and Lp{m,n,X), the 
appropriate stages of image analysis and processing are carried out. 

Image processing 

The images Lc{m,n,X) and Lp{m,n,X) resulting from the initial stage of image processing 
are used for further processing steps. The image Lc{m,n,X) is further used in order to 
isolate the region and the object of interest. The image Lp{m,n,X) is a measurable image 
with respect to the value A. 

Since the aim of the analysis is to determine melanin and haemoglobin quantity in se- 
lected areas of the right hand, an important element is the location of characteristic 
areas. These areas included the individual fingers from Vi to V5 (pollex, index, digitus 
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medius, digitus annularis, digitus minimus manus) and the metacarpus and wrist area 
Ve. Automatic localization of these places (points) from Vi to was made based on 
the pattern shown in Figure 2. Automatic recognition of individual locations (from Vi 
to Ve) for each of the patients was carried out based on a hierarchical approach. The 
image Lc{m,n,X) is reduced to a resolution being 10% of the original resolution, that is 
from the resolution 899 x 1312 pixels to 90 x 131 pixels using the nearest neighbour 
method. The result is the image Li){m,n,X,a) where a is the angle of rotation relative to 
Lc{m,n,X), Then, the angle of the hand inclination a is roughly determined, i.e.: 



a* = argl max! y^y^LD(ni,n,X,a) 



(3) 



and aG(0M80°). 

The obtained result is the value of the rotation angle a* at which the hand image 
must be rotated for each wavelength X so that it is placed in a horizontal position. The 
rotation angle is constant for each X so it does not occur in the formula (3). The image 
Lc{m,n,X) rotated at the angle a is subjected to options for finding characteristic points 
from Vi to Ve. Detection of the position of characteristic points is carried out based on 
the local minima found on the contour of the hand - Figure 3. The contour curve jkiyyi) is 
formed on the basis of the binary image LB{m,n) resulting from binarization of the 
image Lj){m,n,X,a'^) for the threshold p^, determined automatically from the Otsus 
formula [23], i.e.: 

X f ^ ^LD{m,n,X,a) > p, 

LB{m,n) = l V (4) 

I 0 other 



The course of the curve yk{m) was determined on the basis of the position of the first 
rows with the pixel values of "1" in the image LB{m,n) for each column. Local minima, 
which are highlighted in Figure 3, were also determined on this basis. In the present 
case, these are the points V2 to (the big finger is not fully visible). The point is 




Figure 2 Hand pattern used during matching to hyperspectral images. The division into individual 
(points) areas from to V^^ is performed automatically. For each area, a number of the finger from V^} 
to l/^v5 and the centre of mass of the wrist V^^^ are assigned. In the upper part of the image, there is the 
brightness pattern sized lOOx 1321 pixels. The distribution and size of the fingers is carried out based on 
anthropometric data. 
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the centre of mass of the wrist calculated on the basis of the image Lo{m,n) resulting 
from the opening operation of the image LB{m,n), i.e.: 

Lo{m^n) = mm{L]:,{m,n)) (5) 

where SE - square structural element sized 201 x 201. 

The size of the structural element SE was selected on the basis of the maximum 
width of the fingers - for the set distance of the camera and its focal length. In the tests 
performed, the width of the biggest finger was no more than 200 pixels. On this basis, 
the centre of mass of the wrist was determined as the coordinates niy^ and riy^ of the 
point Ve, i.e.: 
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«v6 



(6) 



(7) 



The centre of mass of the wrist is necessary to roughly detect the position of the 
hand relative to the pattern. After this stage, the positions of the individual points of 
fingers, i.e. Vi, V2, V3, V4 and V5, are matched. Matching is performed on the basis of 
minimization of the distance between the individual points of the pattern and the ana- 
lysed image. For the points of the pattern V^i^ V^2> Kvs? Kv4 and V^s> this is mi- 
nimization of the criterion /: 



i=5 



i=5 



(8) 



where m^/ and m^^i - coordinates of the points of the analysed image and the pattern 
/g(1,5). 

The value of the criterion / is calculated for the displacement of the points Vi, V2, V3, 
V4 and V5 in the range of ±100 pixels in the axis of rows and columns. This range is 
sufficient for searching for the minimum value of /. Examples of the obtained values of 
the criterion / are shown in Figure 4. In this case, displacements amounting to -18 
and -3 pixels in the axis of rows and columns respectively were obtained. This value 




A m [pixel] 



-100 



-100 



A n [pixel] 



Figure 4 Graph of changes in the value of the criterion 7 as a function of Am and An. Depending on 
the displacements of tlie points to I/5 (Am and An), tlie value of the criterion J is calculated. One global 
minimum occurring for the displacements -18 and -3 pixels in the axis of rows and columns respectively 
is visible. This value provides information about the need to move the image of the patient so that it 
would better (with respect to the criterion J) match the pattern. 
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informs about the need to move the image of the patient so that it would better (with 
respect to the criterion J) match the pattern. Final adjustment of the positions of the in- 
dividual points of the fingers is performed on the basis of an analogous criterion. How- 
ever, it applies to only one vertex V, A block diagram of the proposed algorithm is 
shown in Figure 5, whereas matching the pattern to the patient's hand is shown in 
Figure 6. The image Lp{m,n,X) corrected in terms of affine transformations is further 
analysed in terms of the content of melanin and haemoglobin. 



h 

(3x3) 



Acquisition of an 
899x1312x800 image 



Filtration 
median filter 



o 

1— 
CI. 

p 



Normalization 



Remove 
background 



Decrease 
the resolution - 10% 



Roughly angle a 



Binarization 



►a* 

-Lb 



Curve yk(m) 



Center of ly 
mass ^ 



Fit Vi V2M3M4 V5yG 



Select fingers 



Calculation of melanin 
and hemoglobin 

t _ 



Statistical analysis 



Results 



P 



(30x30) 



I 

m 

5 



Figure 5 Block diagram of tiie proposed algoritiim for image analysis and processing. The subsequent 
algorithm blocks are profiled to the described issue. In the stage of pre-processing, interference is filtered out of 
the input image after it is reorganized from the file "".dat to the sequence of images. Next, the image is matched 
to the pattern for the subsequent frames. In the final stage, it is analysed in specified areas. 
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Figure 6 Contours of melanin and iiaemogiobin images presented as components R-red and 
G-green respectively. For melanin, it is tine contour formed as tine sum of liyperspectral images in tine 
frequency range A^(450,600) nm, wliereas for liaemoglobin tliis is tine range /\f7e(350,500) nm. Witliin tliese 
ranges, tliere are 189 images Lp{m,n,X} for melanin and 126 images for haemoglobin. The image for melanin 
and the result of automatic segmentation into characteristic areas, described in the paper, are shown on 
top and separation into individual objects at the bottom. 



The performed and described analysis enables automatic and reproducible determin- 
ation of the position of individual areas of the hand. This process allows to perform fur- 
ther analysis of the brightness of the specific wavelengths A. These lengths correspond to 
the maximum emissivity of melanin and haemoglobin. For melanin, this is the frequency 
range A£g(450,600) nm and for haemoglobin it is A/^(397 nm,500 nm) extending into the 
ultraviolet. Within these ranges, there are 189 images Lp{m,n,X) for the melanin and 126 
images for haemoglobin. Accordingly, the image for melanin LE{m,n) was determined as: 

l£(m,«)= L;im,n,X) (9) 

Ag (450,600) 

and analogously for the haemoglobin L^niyn): 

Ag(397,500) 

The results obtained after brightness normalization are shown in Figure 6. The red 
component is the image ^^(m,^^), while the green component is the image The 



Koprowski et al. BioMedical Engineering Online 2014, 13:47 
http://www.bionnedical-engineering-online.conn/content/1 3/1/47 



Page 10 of 15 



blue component was extinguished. The images LE{m,n) and Lnim^n) are the basis for 
further analysis of the results. Additionally, owing to the applied matching to the pat- 
tern, the division into individual fingers and the wrist is further used - Figures 6 and 7. 

Results 

The results of automatic division into the fingers and wrist of the human hand (Figure 6) 
are further used in the tests. These analyses include automatic determination of the 
changes in the spectrum absorption for different frequencies Ag(397,1030) for individual 
fingers. To this end, along each finger, the mean along its symmetry axis (for subsequent 
n) for a given frequency A was calculated. The obtained results for the sample calculations 
of melanin are shown in Figure 8. For subsequent fingers, characteristic changes in the 
intensity of each frequency A are visible. The last graph in Figure 8 shows changes in 




Figure 7 Result of segmentation for individual wavelengths. The sample finger (I/2) was divided into 
six areas from 02,^^7 to 02,a6 for tine left side and from 02,e/ to 02^b6 for the right side. Within these areas, the 
mean value of melanin and its standard deviation and proportions relative to the other areas are calculated. 

V J 
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A/nmJ ™ ° n [pixel} 



V3 




A[nm] 



n fp/xe/J 



V 



n [pfxei] 




n [pixel] 




300 3CD 

n [pixel] 



Figure 8 Graphs of changes in the value of spectrum for wavelengths K for the patient's 
subsequent fingers. The values on the axes are successive columns, wavelength X and brightness for the 
selected wavelength. The last sixth graph shows the change of melanin along all five fingers of the patient. 
The y-value is the mean value of brightness for ^^£(450,600) nm. The colours of individual lines correspond 
to the colours of the fingers shown at the bottom in Figure 6. 



melanin along subsequent fingers. When approaching the wrist, an increase in the inten- 
sity by about 10% to 20% can be observed. The division of one finger (vertex into dif- 
ferent areas is shown at the bottom in Figure 7. The finger was divided into 12 areas, 6 on 
the side "A" and 6 on the side "B" of the finger - that is from 02,ai to 02,a6 for the side A 
and from 02,bi to 02,b6 for the side B respectively - Figure 7. These areas are automatic- 
ally scaled depending on the size of the finger and other camera settings which may affect 
the size of the object. The results of the average brightness intensity and standard 
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deviation of the mean for each area are presented in Table 1. Between the sides A and B, 
brightness changes are not greater than 1% {02,a2, 02,b2 and 02,a3> 02,33)- The standard 
deviation of the mean is the highest for the areas of distal phalanges - areas 02,ai and 
O^BP The situation is similar for haemoglobin - Table 2, The standard deviation of the 
mean is a bit smaller than in the case of melanin - it oscillates around the brightness 
value of 0.02. Higher brightness values exist for other wavelengths not included in the 
calculation of melanin or haemoglobin - Figure 8. 

Comparison with other authors' results 

The known results and methods presented by other authors can be divided into: 

• Issues related to hyperspectral imaging involving image analysis and processing and 

• Methods of analysis and processing of hand images that can be used in 
hyperspectral imaging. 

In the first case, hyperspectral imaging was used, inter alia, to analyse the saturation 
of Hb02 in people of different races (Zuzak [14,15]). Sampling from the hyposthenia 
region of African- American controls, the area within the square on the image, the per- 
centage of skin Hb02 was 77.5 ± 0.2%, which is similar to the skin Hb02 percentage of 
78.2 ± 0.2% in healthy Caucasian subjects. In contrast, the percentage of skin Hb02 in 
patients was significantly smaller and amounted to 61.0 ± 0.2% (p < 0.001). Percent 
renal parenchymal oxyhaemoglobin was also analysed by Liu [24]. In [13], there are also 
other results of analysis which is related only to the analysis of the whole ROI (region 
of interest) marked manually by an operator. The use of hyperspectral imaging to as- 
sess the time of the bruise formation is also interesting - Stam [25]. The inaccuracy 
found is 2.3% for fresh bruises and 3 to 24% for bruises up to 3 days old. In Authors' 
conclusion [25], colour inhomogeneity of bruises can be used to determine their age. 
For example, the experiment results presented in [13] show that the hyperspectral 
based method has the potential to identify the spinal nerve more accurately than the 
traditional method as the new method contains both the spectral and spatial informa- 
tion of nerve sections. There are many other areas of medicine where manual or semi- 
automatic marking of regions of interest is used - analysis of vibrational Filik J. [26], 
laparoscopic digital light processing - Olweny EO. [27], blood stains at the crime scene - 
Edelman G. [28], prostate cancer detection - Akbari H. [29], histopathological examination 
of excised tissue - Vasefi F. [30] diabetic foot ulcer - Yudovsky D. [31], cancer detection - 
Akbari H. [32], and others. 

In the other case, these are morphological operations used in the classification of 
various types of artefacts visible in the image [1,2,5,6]. Other methods for classification 
are also used such as SVM (support vector machines) [4], Gauss-Markov model [7] or 



Table 1 The percentage of melanin for the sample finger (vertex V2) and its standard 
deviation (STD) of the mean 
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Table 2 The percentage of haemoglobin for the sample finger (vertex V2) and its 
standard deviation (STD) of the mean 





02,A1 


02,A2 


02,A3 


02,A4 


02,A5 


02,A6 


02,B1 


02,B2 


02,B3 


^2,84 


02,B5 


02,B6 


Brightness 


0.17 


0.14 


0.15 


0.16 


0.15 


0.15 


0.17 


0.14 


0.15 


0.16 


0.15 


0.15 
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0.04 


0.02 


0.02 


0.01 


0.01 


0.01 


0.04 


0.02 


0.02 


0.01 


0.01 


0.01 



wavelet method [9]. In the study of Dicker et al. [16], spectral library was gener- 
ated with 12 unique spectra, which was used to classify specimens where sample 
preparation was varied. Using a CT of 0.99 left large areas of the tissues unclassi- 
fied. Lowering the minimum correlation coefficient to 0.99 enabled all the samples 
to achieve >85% classification. The results referred mainly to the analysis of a hyper- 
spectral image by analyzing the histograms obtained. Liu Z. et al. [12] propose a novel 
tongue segmentation method that uses hyperspectral images and the SVM. The pre- 
sented segmentation of the tongue allows to obtain reproducible and quantitative re- 
sults. Benediktsson J. et al. [2] present results for a sequential use of morphological 
opening and closure for the increasing size of the structural element. This method- 
ology is similar to the use of conditional erosion and dilation as in [3]. In turn, Rellier 
G. et al. [7] propose a probabilistic vector texture model, using a Gauss-Markov ran- 
dom field (MRF). The MRF parameters allow the characterization of different hyper- 
spectral textures. 

In conclusion, the well-known studies related to using the methods for the analysis 
and processing of images into hyperspectral images is dominated by morphological 
analysis. Segmentation into specific areas refers to simple objects such as the tongue. 
In each case the methods described are profiled to a particular application. Therefore, 
the approach to the analysis of the hand proposed in this paper is an extension of these 
methods into a new area and new dedicated analysis methodology. 

Summary 

The paper presents a method for the analysis of melanin and haemoglobin in the area 
of the human hand. The characteristics of the described method are as follows: 

• Repeatability of measurements owing to limiting of operator s participation in the 
study, 

• Full automatic operation of the algorithm - the arguments for each algorithm func- 
tion are set once when first starting the algorithm - as they only depend on the 
type of the multispectral camera used, 

• Possibility of any quantitative (not qualitative) assessment of the amount of melanin 
or haemoglobin in any area of the hand, 

• Possibility of automatic comparison of the results of any area of the finger with 
other areas or other study of the same person in therapy/disease monitoring, 

• Time analysis of a single image sequence does not exceed 100 ms when using Intel 
Core i5 CPU M460 @2.5 GHz 4 GB RAM. 

The discussed methodology of hyperspectral image analysis and processing does not 
fully cover the issue. The presented algorithm for image analysis and processing can 
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also be created by using the techniques from spectral methods [33,34], optical image 
analysis [35], microscopic analyses [36,37] and others [38-41]. In future work, the au- 
thors intend to analyse reproducibility of results for a larger number of patients using 
different types of hyperspectral cameras operating in the same spectral range. The im- 
pact of lighting and calibration method is equally interesting, which will be the subject 
of the authors' future work. 
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